A Graduated Nonconvex Regularization for Sparse High Dimensional Model Estimation

Many high dimensional data mining problems can be formulated as minimizing an empirical loss function with a penalty proportional to the number of variables required to describe a model. We propose a graduated non-convexification method to facilitate tracking of a global minimizer of this problem. We prove that under some conditions the proposed regularization problem using the continuous piecewise linear approximation is equivalent to the original 0 l regularization problem. In addition, a family of graduated nonconvex approximations are proposed to approximate its 1 l continuous approximation. Computational results are presented to illustrate the performance.